Systems, devices and methods for automatic microscope focus

ABSTRACT

An automatic focus system for an optical microscope that facilitates faster focusing by using at least two offset focusing cameras. Each offset focusing camera can be positioned on a different side of an image forming conjugate plane so that their sharpness curves intersect at the image forming conjugate plane. Focus of a specimen can be adjusted by using sharpness values determined from images taken by the offset focusing cameras.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/207,727, filed Dec. 3, 2018, which is a continuation of U.S. patentapplication Ser. No. 15/967,802, filed May 1, 2018, now U.S. Pat. No.10,146,041, each of which is hereby incorporated by reference herein inits entirety.

TECHNICAL FIELD

The present disclosure relates to image-based mechanisms for automaticmicroscope focus.

BACKGROUND

Most specimens that are observed with a microscope have small variationsin height across their surfaces. While these variations are frequentlynot visible to the human eye, they can cause images of a portion of aspecimen captured by a microscope to be out of focus.

The range in which a microscope can create a usable focused image isknown as the depth of field. The microscope must keep a portion of aspecimen within its depth of field to generate useful images. However,when transitioning from observing a first portion of a specimen toobserving a second portion of the specimen, the small variations inheight of the specimen may cause the second portion to be outside thedepth of field.

Different sharpness measurements such as image contrast, resolution,entropy and/or spatial frequency content, among others, can be used tomeasure the quality of focus of images captured by a microscope.Generally, when a specimen is in focus, the captured image will exhibitthe best sharpness quality (e.g., large contrast, a high range ofintensity values and sharp edges). The different sharpness measurementsthat can be used to determine when a specimen is in focus usuallyrequire capturing a series of images at different distances between amicroscope objective lens and the specimen (i.e., the relative Zposition), and measuring the sharpness of the captured images until theimage appears in focus. Because measuring the sharpness value of thespecimen at a relative Z position will generally not indicate thedirection of adjustment (i.e., whether to increase or decrease thedistance) required to bring the specimen in focus, a greater number ofimages and adjustments are generally required to focus an image than ifthe direction of adjustment were known. This increases the totalmicroscopic scan time of each specimen, which can be detrimental in highthroughput scanning applications.

Also, because sharpness measurements can have relatively constant valuesover relative Z positions near the in-focus position, simply looking fora peak value of a sharpness curve may not accurately identify thein-focus position.

Accordingly, new mechanisms for automatic microscope focus aredesirable.

SUMMARY

In accordance with some embodiments, systems, devices and methods forautomatic microscope focus are provided. In some embodiments, systemsfor automatically focusing a microscope are provided, the systemscomprising: an objective; a stage for positioning a specimen on a firstimage forming conjugate plane; a first focusing camera, configured forfocusing, positioned on a first side of a second image forming conjugateplane at a first offset distance; a second focusing camera, configuredfor focusing, positioned on a second side of the second image formingconjugate plane at a second offset distance; wherein the first offsetdistance and the second offset distance are determined so that sharpnessmeasurements for images of the specimen captured by each of the firstfocusing camera and the second focusing camera, at a same distancebetween the objective and the stage, are equal at the second imageforming conjugate plane; a primary illumination source; an imagingdevice positioned on a third image forming conjugate plane; and ahardware processor coupled to the first focusing camera and the secondfocusing camera that is configured to determine that the specimen is infocus when a sharpness value of the specimen using the first focusingcamera is equal to a sharpness value of the specimen using the secondfocusing camera.

In some embodiments, methods are provided for automatically focusing amicroscope having at least an objective, a stage for positioning aspecimen on a first image forming conjugate plane, a first focusingcamera, configured for focusing, positioned on a first side of a secondimage forming conjugate plane at a first offset distance, a secondfocusing camera, configured for focusing, positioned on a second side ofthe second image forming conjugate plane at a second offset distance, aprimary illumination source, an imaging device positioned on a thirdimage forming conjugate plane, the method comprising: setting the firstoffset distance and the second offset distance so that sharpnessmeasurements for images of the specimen captured by each of the firstfocusing camera and the second focusing camera, at a same distancebetween the objective and the stage, are equal at the second imageforming conjugate plane; and determining that the specimen is in focuswhen a sharpness value of the specimen using the first focusing camerais equal to a sharpness value of the specimen using the second focusingcamera.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A shows an example of an automatic focus system using twoillumination sources in accordance with some embodiments of thedisclosed subject matter.

FIG. 1B shows an example of an automatic focus system using oneillumination source in accordance with some embodiments of the disclosedsubject matter.

FIG. 2A shows an example of an illumination unit using two illuminationsources in accordance with some embodiments of the disclosed subjectmatter.

FIG. 2B shows an example of an illumination unit using one illuminationsource in accordance with some embodiments of the disclosed subjectmatter.

FIG. 3A shows an example of a focusing unit of an automatic focus systemusing two illumination sources in accordance with some embodiments ofthe disclosed subject matter.

FIG. 3B shows an example of a focusing unit of an automatic focus systemusing one illumination sources in accordance with some embodiments ofthe disclosed subject matter.

FIG. 4 shows an example of a sharpness curve for an imaging device inaccordance with some embodiments of the disclosed subject matter.

FIG. 5 shows an example of sharpness curves for two offset focusingcameras in accordance with some embodiments of the disclosed subjectmatter.

FIG. 6 shows an example of a flow chart of a process for performingautomatic focus using an automatic focus system, such as the systemillustrated in FIGS. 1A and 1B, in accordance with some embodiments ofthe disclosed subject matter.

FIG. 7 shows an example of a flow chart of a process for finding animage-forming conjugate plane and calibrating two offset focusingcameras, using an automatic focus system, such as the system illustratedin FIGS. 1A and 1B, in accordance with some embodiments of the disclosedsubject matter.

DETAILED DESCRIPTION

In accordance with some embodiments of the disclosed subject matter,mechanisms (which can include systems, methods, devices, apparatuses,etc.) for automatic microscope focus of specimens are provided.

FIGS. 1A and 1B illustrate examples of an automatic focus system 100according to some embodiments of the disclosed subject matter. At a highlevel, the basic components of automatic focus system 100, according tosome embodiments, include an illumination unit 200 for providing light,a focusing unit 300 for finding the in-focus plane of a specimen, anilluminator 13, an imaging device 5, an objective 25, a stage 30, and acontrol system 108 comprising hardware, software, and/or firmware.

Automatic focus system 100 can be implemented as part of any suitabletype of microscope. For example, in some embodiments, system 100 can beimplemented as part of an optical microscope that uses transmitted lightor reflected light. More particularly, system 100 can be implemented aspart of the nSpec® optical microscope available from NanotronicsImaging, Inc. of Cuyahoga Falls, Ohio. Although the followingdescription refers to a reflected light illuminator 13, the mechanismsdescribed herein can be a part of microscopes that do not use areflected light illuminator.

According to some embodiments, system 100 can include, one or moreobjectives 25. The objectives can have different magnification powersand/or be configured to operate with brightfield/darkfield microscopy,differential interference contrast (DIC) microscopy and/or any othersuitable form of microscopy including fluorescence. The objective and/ormicroscopy technique used to inspect a specimen can be controlled bysoftware, hardware, and/or firmware in some embodiments.

In some embodiments, a fine focus actuator 23 can be used to driveobjective 25 in a Z direction towards and away from stage 30. Fine focusactuator 23 can be designed for high precision and fine focus adjustmentof objective 25. Fine focus actuator 23 can be a stepper motor, servomotor, linear actuator, piezo motor, and/or any other suitablemechanism. For example, in some embodiments, a piezo motor can be usedand can drive the objective 0 to 50 micrometers (μm), 0 to 100 μm, or 0to 200 μm, and/or any other suitable range(s) of distances.

In some embodiments, an XY translation stage can be used for stage 30.The XY translation stage can be driven by stepper motor, servo motor,linear motor, piezo motor, and/or any other suitable mechanism. The XYtranslation stage can be configured to move a specimen in the X axisand/or Y axis directions under the control of any suitable controller,in some embodiments.

In some embodiments, focus mechanism 32, comprising actuator 35, can beused to adjust stage 30 in a Z direction towards and away from objective25. Actuator 35 can be used to make coarse focus adjustments of, forexample, 0 to 5 mm, 0 to 10 mm, 0 to 30 mm, and/or any other suitablerange(s) of distances. Actuator 35 can also be used to move stage 30 upand down to allow specimens of different thicknesses to be placed on thestage. Actuator 35 can also be used in some embodiments to provide finefocus of, for example, 0 to 50 μm, 0 to 100 μm, 0 to 200 μm, and/or anyother suitable range(s) of distances. In some embodiments, focusmechanism 32 can also include a location device 33. The location devicecan be configured to determine a position of stage 30 at any suitablepoint in time. In some embodiments, any suitable position (e.g., theposition of the stage when a specimen is in focus) can be stored in anysuitable manner and later used to bring the stage back to that position,even upon reset and/or power cycling of autofocus system 100. In someembodiments, the location device can be a linear encoder, a rotaryencoder or any other suitable mechanism to track the absolute positionof stage 30 with respect to the objective.

In some embodiments, automatic focus system 100, when properly focusedand aligned, can use a set of conjugate focal planes, for example animage-forming conjugate set (as shown in FIGS. 1A and 1B), that occuralong the optical pathway through the microscope. Each plane within theimage-forming conjugate set is conjugate with the others in that setbecause the planes are simultaneously in focus and can be viewedsuperimposed upon one another when observing specimens through themicroscope. The set of image-forming conjugate planes used in automaticfocus system 100 can include a focusing unit image-forming conjugateplane 80 (“focusing conjugate plane 80”), an imaging deviceimage-forming conjugate plane 6 (“imaging conjugate plane 6”), anillumination unit image-forming conjugate plane 54 (“illuminationconjugate plane 54”), a field diaphragm (F-stop) image-forming conjugateplane 21 (“field diaphragm conjugate plane 21”) and a specimenimage-forming conjugate plane 8 (“specimen conjugate plane 8”). Allreferences herein to positioning a first offset focusing camera 70, asecond offset focusing camera 72, and/or an imaging camera 5 (whenimaging device 5 is a camera), on or offset to an image-formingconjugate plane, refer to positioning the sensors within cameras 5, 70and/or 72 on or offset to the image-forming conjugate plane. In someembodiments, illumination conjugate plane 54 and/or field diaphragmconjugate plane 21 can be omitted.

In some embodiments, imaging device 5 can be a camera that includes animage sensor that is positioned on image-forming conjugate plane 6 ofautomatic focus system 100. Imaging device 5 can be used to captureimages of a specimen, e.g., once control system 108 determines that thespecimen is in focus. The image sensor can be, for example, a CCD, aCMOS image sensor, and/or any other suitable electronic device thatconverts light into one or more electrical signals. Such electricalsignals can be used to form images and/or video of a specimen. In someembodiments, imaging device can be replaced with an ocular or aneyepiece that is used to view a specimen.

In some embodiments, control system 108, comprising controller 110 andcontroller interface 107, can control any settings of the components ofautomatic focus system 100 (e.g., actuators 35 and 23, primaryillumination source 65, secondary illumination source 40, offsetfocusing cameras 70 and 72, stage 30, focusing pattern 55, imagingdevice 5 and objective 25, etc.), as well as communications, operations(e.g., taking images, turning on and off an illumination source, movingstage 30 and objective 25, storing different values associated with aspecimen) and calculations (e.g., sharpness calculations) performed by,and between, components of the automatic focus system. Control system108 can include any suitable hardware (which can execute software insome embodiments), such as, for example, computers, microprocessors,microcontrollers, application specific integrated circuits (ASICs),field-programmable gate arrays (FGPAs) and digital signal processors(DSPs) (any of which can be referred to as a hardware processor),encoders, circuitry to read encoders, memory devices (including one ormore EPROMS, one or more EEPROMs, dynamic random access memory (“DRAM”),static random access memory (“SRAM”), and/or flash memory), and/or anyother suitable hardware elements. In some embodiments, individualcomponents within automatic focus system 100 can include their ownsoftware, firmware, and/or hardware to control the individual componentsand communicate with other components in automatic focus system 100.

In some embodiments, communication 120 between the control system (e.g.,controller 110 and controller interface 107) and the components ofautomatic focus system 100 can use any suitable communicationtechnologies, such as analog technologies (e.g., relay logic), digitaltechnologies (e.g., RS232, ethernet, or wireless), network technologies(e.g., local area network (LAN), a wide area network (WAN), theInternet) Bluetooth technologies, Near-field communication technologies,Secure RF technologies, and/or any other suitable communicationtechnologies.

In some embodiments, operator inputs can be communicated to the controlsystem using any suitable input device 105 (e.g., a keyboard, mouse orjoystick).

FIG. 2A shows the general configuration of an embodiment of anillumination unit of the automatic focus system, in accordance with someembodiments of the disclosed subject matter. The illumination unit 200can include two illumination sources, for example a primary illuminationsource 65 and a secondary illumination source 40. The illuminationsources can provide light beams in ranges of wavelengths that aredifferent from each other. In other embodiments, the illumination unit200 can include only a primary illumination source 65, as shown, forexample, in FIG. 2B.

In some embodiments, for example, primary illumination source 65provides a light beam having a wavelength in the range of 451 to 750nanometers (nm), while the secondary illumination source 40 provides alight beam having a wavelength that is higher or lower than the range ofwavelengths used for the primary source. For example, the wavelengthrange of the primary illumination source 65 can be in the range of 550to 750 nm and the wavelength range of the secondary illumination sourcecan be in the range of 400 to 450 nm. Light of any wavelength range canbe used for primary illumination source 65 as long as the value of therange is known and can be separated from other wavelengths using knownfiltering techniques. Similarly, light of any wavelength range can beused for secondary illumination source 40, as long as the light is notin the same wavelength range as primary illumination source 65.

In some embodiments, as shown in FIG. 1A, primary illumination source 65is positioned so that its light is transmitted in a horizontal directiontowards illuminator 13. Primary illumination source 65 can include afocusing lens 49 (e.g., a double convex lens) for focusing the primarylight beam. The secondary illumination source 40 can be positioned at asuitable distance below a focusing pattern 55 located on image-formingconjugate plane 54.

In some embodiments, focusing pattern 55 can be formed from opaquematerial, with a pattern cut out of the material. The cutout section ofthe material allows light to pass through to specimen conjugate plane 8,while the opaque material section blocks light from passing through. Inother embodiments, focusing pattern 55 can be formed from clear materialsuch as clear glass or clear plastic that has an opaque pattern thereonwhich causes an image to be projected on the specimen conjugate plane bylight passing through the clear glass or plastic. In furtherembodiments, focusing pattern 55 can be digitally controlled (e.g., aspecial light modulator).

The diameter of focusing pattern 55 (e.g., 5 mm) can be adjusted so thata projection of focusing pattern 55 is smaller than the field of view(FOV) of offset focusing cameras 70 and 72. Focusing pattern 55 can beany suitable geometric shape for example, a circle, rectangle, triangle,or hexagon and can be projected on any area of the FOV. Focusing pattern55 can also include a series of discrete openings, so that when light istransmitted through the discrete openings, the lines and spaces areprojected across the field of view. In some embodiments, focusingpattern 55 can be customized for a specimen. In some embodiments, thelocation of primary illumination source 65 and secondary illuminationsource 40 can be switched.

In some embodiments, automatic focus system 100 can be configured sothat light from secondary illumination source 40 is continuouslytransmitted through focusing pattern 55 in order to continuously projectthe focusing pattern image on a specimen that can be captured by offsetfocusing cameras 70 and 72. The continuous projection of the focusingpattern image can facilitate sharpness focus of a specimen, especiallyfor transparent specimens or for specimens that lack any visuallyrecognizable features. Focusing pattern 55 can be used instead of, or inaddition to, a field diaphragm, for sharpness focusing. For example,automatic focus system 100, in addition to focusing pattern 55, can alsoinclude a field diaphragm (F-stop) 14 that can be located in theilluminator 13. Field diaphragm 14 can also be positioned on animage-forming conjugate plane of automatic focus system 100. In someembodiments, field diaphragm 14 controls the diameter of light emittedby illumination source 65 and 40 and transmitted to objective 25. Morespecifically, in some embodiments, by reducing the size of the fielddiaphragm, the diameter of the light passing through is reduced. Thiscreates a dark outline around the image of the specimen received byoffset focusing cameras 70 and 72 and can be used to adjust the focus ofthe specimen (e.g., by moving the specimen and objective closer togetheror farther apart). At the point of greatest measured sharpness, thespecimen is considered to be in-focus and the field diaphragm can beopened to a larger size to allow imaging of the specimen by imagingdevice 5. Reducing the field diaphragm and returning it to its originalsize, however, takes time (e.g., 2-5 seconds) and can slow down thescanning process and throughput.

Focusing pattern 55 can be positioned on an any suitable image-formingconjugate plane of automatic focus system 100 (e.g., above secondaryillumination source 40 (as shown in FIGS. 1A and 2A), or at fielddiaphragm 14), as long as an appropriate filter (e.g., filter 11) isused, when necessary, to make sure that focusing pattern 55 is notprojected onto imaging device 5. For example, if focusing pattern 55 ispositioned on the field diaphragm 14 image forming conjugate plane, inplace of field diaphragm 14, then a filter would be necessary. In someembodiments, a band filter can be located on the field diaphragm imageforming conjugate plane (in place of field diaphragm 14) and a focusingpattern in the form of a pattern cutout can be created in the bandfilter. More specifically, a band filter can be selected that transmitslight in the same wavelength range of primary illumination source 65(e.g., greater than 450 nm) and blocks light in the same wavelengthrange of secondary illumination source 40 (e.g., less than or equal to450 nm), except in the focusing pattern 55 region. In other words, lightin the same wavelength range of secondary illumination 40 source wouldbe blocked except in the region of focusing pattern 55, which wouldallow the light from secondary illumination 40 to be transmitted throughto offset focusing cameras 70 and 72.

In some embodiments, when using a single illumination source, focusingpattern 55 can be digitally controlled (e.g., using a special lightmodulator) and can be located, for example, on the field diaphragm imageforming conjugate plane of automatic system 100. More specifically, insome embodiments, when digitally controlled, focusing pattern 55 can becontrolled to be disabled at predetermined intervals, so that imagingdevice 5 can form electrical signals representing an image and/or videoof the specimen without interference from focusing pattern 55 (at fullfield of view). In some embodiments, imaging device 5 can be configuredso that it does not form such electrical signals of a specimen whenfocusing pattern 55 is enabled.

Note that, in some embodiments, any suitable illumination source(s) canbe used with illumination unit 200, such as a 400 nm ultravioletcollimated light-emitting diode (LED) for secondary illumination source40 and a 5500 K white light collimated LED for primary illuminationsource 65. In some embodiments, lasers or fluorescent light can be usedfor primary illumination source 65 and/or secondary illumination source40.

In some embodiments focusing lens 45 (e.g., a 60 mm focal lengthbioconvex lens) can be placed at a suitable distance between thesecondary illumination source 40 and focusing pattern 55. Further,another focusing lens 47 can be placed at a suitable distance on theother side of focusing pattern 55. In some embodiments, the distance ofthe lenses 45 and 47 from focusing pattern 55 can be based on theoptical characteristics of the microscope to ensure the focusing of thelight and positioning of focusing pattern 55 to be in a conjugateimage-forming plane.

In some embodiments that use two illumination sources, a dichroic 60 isplaced in the optical pathway of both primary illumination source 65 andsecondary illumination source 40 before the light travels to illuminator13. Dichroic, as used herein, can refer to mirrors, beam splitters,filters or beam combiners that transmits light of a known, specifiedwavelength and combines the transmitted light with a light of anotherknown, specified wavelength. Note that a combination of theaforementioned devices can be used to reflect and transmit the desiredillumination sources and wavelengths. In some embodiments, a dichroichaving a specific cut-off wavelength is selected in order to reflect thewavelengths of light emitted by secondary illumination source 40 and toallow the wavelengths of light emitted from primary illumination source65 to pass through. For example, if secondary illumination source 40emits light in a wavelength range of 400-450 nm and primary illuminationsource 65 emits light in a wavelength range of 550-750 nm, then a 450 nmcutoff dichroic (i.e., a dichroic that reflects light with a wavelengthof 450 nm and below and allows light with a wavelength greater than 450nm to pass through thereby combining the beams) can be used to reflectlight from secondary illumination source 40 and to allow light fromprimary illumination source 65 to pass through. Dichroic 60 can bedesigned for a 45° angle of incidence, so that rejected light fromsecondary illumination source 40 is reflected at an angle of 90° andtravels parallel to the light path from primary illumination source 65.

In some embodiments, primary illumination source 65 can be the lightsource used for imaging device 5 and secondary illumination source 40can be the light source used for imaging a specimen on focusing sensors71 and 73 of offset focusing cameras 70 and 72 (as shown in FIGS. 1A and3A).

Note that, in some embodiments any suitable dichroic, illuminator,illumination source, focusing lens, sensor and focusing pattern can beused with illuminating unit 200. In some embodiments, any suitablearrangement of these components can be used with illuminating unit 200.In some embodiments, the components of illuminating unit 200 can bemounted to illuminator 13 in any suitable manner, such as by using guiderods in a similar manner to how offset focusing camera 72 is shown asbeing mounted to focusing housing 18 in FIG. 3 (described below), inorder to allow variable geometry.

FIGS. 3A and 3B show an example of a general configuration of anembodiment of a focusing unit of the automatic focus system, inaccordance with some embodiments of the disclosed subject matter. Thefocusing unit 300 can include two cameras: a first offset focusingcamera 70 and a second offset focusing camera 72. These cameras caninclude, for example, a charged coupled device (CCD) image sensor, aCMOS image sensor, another form of image sensor, a video sensor and/orany other suitable sensor that can form electrical signalsrepresentative of an image and/or video of a specimen. In someembodiments, such electrical signals can be stored and analyzed bycontrol system 108.

The focusing unit 300 can be mounted in an area between illuminator 13and imaging microscope tube lens 10. This area can be referred to asinfinity space. In some embodiments, the focusing unit 300 can bemounted in other locations using appropriate components to adapt theselected location to the optical characteristics of the system.

First offset focusing camera 70 can include a sensor 71 that ispositioned at an offset distance f1 to focusing conjugate plane 80. Theoffset distance f1 can be either be in the positive direction 81 or inthe negative direction 79. A second offset focusing camera 72 caninclude a sensor 73 that can be positioned at an offset distance f2 tofocusing conjugate plane 80. The offset distance f2 can be either be inthe positive direction 81 or in the negative direction 79.

As shown in FIGS. 3A and 3B, first offset focusing camera 70 and secondoffset focusing camera 72 are positioned on different sides of focusingconjugate plane 80. For example, in some embodiments, in which focusingconjugate plane 80 is vertical, first offset focusing camera 70 can bepositioned to the left of focusing conjugate plane 80 and second offsetfocusing camera 72 can be positioned to the right of conjugate plane 80(as shown in FIGS. 3A and 3B), and vice versa. Further, second offsetfocusing camera 72 can be located above or below first offset focusingcamera 70. In some embodiments (not shown), in which focusing conjugateplane 80 is horizontal, offset focusing camera 70 can be positionedabove focusing conjugate plane 80 and offset focusing camera 72 can bepositioned below focusing conjugate plane 80, and vice versa. Further,second offset focusing camera 72 can be located to the right or to theleft of offset focusing camera 70. As discussed in connection with FIGS.5-7 , offset focusing cameras 70 and 72 can be positioned so thatsharpness values for images and/or video of a specimen captured byoffset focusing cameras 70 and 72 will be the same at focusing conjugateplane 80.

First offset focusing camera 70 can be movable along guide rods 77 orany other suitable structure in order to adjust an offset distance offirst offset focusing camera 70. Second offset focusing camera 72 can bemovable along guide rods 76 or any other suitable structure in order toadjust an offset distance of second offset focusing camera 72.

Focusing unit 300 can also include two focusing lenses 24 and 22.Focusing lens 22 can be placed in the same horizontal optical pathway asfirst offset focusing camera 70 and focusing lens 24 can be placed inthe same horizontal optical pathway as second offset focusing camera 72.In some embodiments, focusing lenses 22 and 24 achieve the same focaldistance as microscope tube lens 10, to ensure that sensors 71 and 73are each in focus when they are positioned on focusing conjugate plane80. Microscope tube lens 10 can include a lens (not shown) for focusingan image of a specimen on imaging conjugate plane 6, so that thespecimen is in focus when an image sensor or ocular is positioned onimage conjugate plane 6 of automatic focus system 100.

Note that in some embodiments, lenses 22 and 24 can be double convexlenses or any other suitable type lenses. In some embodiments, the focallength of the lenses can be based on the optical characteristics of themicroscope.

As also shown in FIG. 3A, in some embodiments that include twoillumination sources (as represented by the pair of longer dashed lines62 and shorter dashed lines 63), focusing unit 300 can also include acutoff dichroic 15 that is positioned above illuminator 13 in theoptical pathway of the light reflected off a specimen. The dichroic 15can be positioned so that the light reflected off the specimen that isbelow the cutoff of the dichroic is reflected at an angle of 90° towardsfirst offset focusing camera 70. A dichroic having a specific cut-offwavelength can be used in order to reflect the wavelengths of lightemitted by secondary illumination source 40 (the “focusing beam”). Forexample, if the focusing beam is in the range of 400 to 450 nm, then a450 nm cut-off filter can be used with focusing unit 300 in order toreflect the focusing beam towards first offset focusing camera 70. Insome embodiments, dichroic 15 is used only in embodiments that includesecondary illumination source 40.

In embodiments including two illumination sources, a cut-off filter 17can be positioned between dichroic 15 and beam splitter 26 to filter outany light coming from primary illumination source 65 (the “imagingbeam”). For example, if imaging beam has a wavelength in the range of450 nm and above, then a 450 nm cutoff filter can be used to filter outthe imaging beam and prevent the imaging beam from transmitting light tofocusing cameras 70 and 72. In other embodiments, two cut-off filterscan be used and each filter can be placed, for example, in front of orbehind lenses 22 and 24.

In embodiments that include only a single illumination source, as shownin FIG. 3B (as represented by lines 62) beam splitter 16, can bepositioned above illuminator 13 in the optical pathway of the lightreflected off a specimen. Beam splitter 16 can be, for example, a 50/50beam splitter designed to send 50% of the light from primaryillumination source 65 to offset focusing cameras 70 and 72, and 50% ofthe light from primary illumination source 65 to imaging device 5.

In some embodiments, as shown in FIGS. 3A and 3B, focusing unit 300 caninclude a beam splitter 26 that can be positioned between dichroic15/beam splitter 16 and first offset focusing camera 70. Beam splitter26 can be, for example, a 50/50 beam splitter designed to send 50% ofthe focusing light beam to first offset focusing camera 70 and 50% ofthe focusing light beam to second offset focusing camera 72. A mirror 28can be placed at a distance directly above beam splitter 26 and can bedesigned to direct the beam of light from beam splitter 26 to secondoffset focusing camera 72.

Note that, in some embodiments any suitable dichroic, focusing camera,focusing lens, mirror, image sensor, beam splitter and cut-off filtercan be used with focusing unit 300. In some embodiments, any suitablearrangement of these components can be used with focusing unit 300. Thecomponents of focusing unit 300 can be mounted to guide rods or anyother suitable structure for connecting the components.

FIGS. 1A and 1B shows example optical pathways, represented by a singleor pair of dashed lines 62 and 63, for automatic focus system 100, inaccordance with some embodiments of the disclosed subject matter. Asshown in FIG. 1A automatic focus system 100 can be configured so thatthe light emitted from secondary illumination source 40 (the “focusingbeam (FB),” as represented by the shorter dashed lines 63) is projectedonto specimen S and then reflected to offset focusing cameras 70 and 72.Autofocus system 100 can also be configured so that light emitted fromprimary illumination source 65 (the “imaging beam (IB),” as representedby the longer dashed lines 62) is projected onto specimen S and thenreflected to imaging device 5.

More specifically, in embodiments that use two illumination sources,focusing beam 62 can travel from illumination source 40 through focusingpattern 55 to dichroic 60. Dichroic 60 can reflect focusing beam 62towards illuminator 13. The imaging beam can travel from primaryillumination source 65, pass through dichroic 60 to combine with thefocusing beam.

The combined beam (if using two illumination sources) can then travelthrough illuminator 13 to prism 20. Prism 20 can reflect the lightcoming from the illumination sources at 90° downwards through anosepiece and objective 25 to a specimen S. Specimen S can reflect thecombined or single beam upwards through objective 25, which is thentransmitted through prism 20 towards dichroic 15. Dichroic 15 (if usingtwo illumination sources) can separate the transmitted beam back intoimaging beam 62 and focusing beam 63 by, for example, reflecting thewavelengths of the light from secondary illumination source 40 towardsoffset focusing cameras 70 and 72 and by allowing the wavelengths of thelight from primary illumination source 65 to pass through towards camera5.

In embodiments that include two illumination sources (as shown in FIGS.1A, 2A and 3A), focusing beam 63 that is reflected by dichroic 15 canpass through cutoff filter 17 to remove any light above the cutoffwavelength. Focusing beam 63 can then travel to beam splitter 26. Beamsplitter 26 can send, for example, 50% of focusing beam 63 towards firstoffset focusing camera 70 by directing the light through focusing lens22 located in focusing housing 18. From there focusing beam 63, cantravel to a light sensor 71 in first offset focusing camera 70. Theother 50% of focusing beam 63 can be directed by beam splitter 26upwards towards mirror 28. Mirror 28 can reflect focusing beam 63towards focusing lens 24 locating in focusing housing 19. From therefocusing beam 63 can be directed to sensor 73 in second offset focusingcamera 72.

In some embodiments that include two illumination sources (as shown inFIGS. 1A, 2A and 3A), imaging beam 62 that passes through dichroic 15can pass through an optical filter 11 (e.g., a filter that transmitsonly the wavelengths from the imaging beam), up through microscope tubelens 10, and to imaging device 5.

In embodiments that include a single illumination source, as shown inFIG. 1B, autofocus system 100 can be configured so that light emittedfrom primary illumination source 65 is projected onto specimen S andthen reflected to offset focusing camera 70 and 72 and imaging device 5.More specifically, beam splitter 16, can be positioned above illuminator13 in the optical pathway of the light reflected off the specimen S.Beam splitter can direct, for example, 50% of the light reflected off ofspecimen S to offset focusing cameras 70 and 72, and 50% to imagingdevice 5.

In some embodiments, the specimen can be brought into focus usingimaging device 5 by moving the objective and the stage closer togetheror farther apart along a Z axis (as shown in FIGS. 1A and 1B) todifferent relative Z positions. In some embodiments, focus can beadjusted using a coarse focus adjustment, which spans a wide range ofZ-positions (such as 500 μm to 2500 μm, or any other suitable range),and/or using a fine focus adjustment, which spans a more-narrow range ofZ-positions (such as 1400 μm to 1600 μm, or any other suitable range).The coarse focus adjustment can be made by actuator 35, in someembodiments. The fine focus adjustment can be made by fine focusactuator 23, in some embodiments.

In some embodiments, a relative sharpness value of an image formed fromlight impacting an image sensor can be used as an indicator of thequality of focus of the specimen. In some embodiments, the higher therelative sharpness value, the more in focus the image is. In someembodiments, as shown in FIG. 4 , a plot of relative sharpness valuesover different relative Z positions can have a distinct peak (this canbe a single point or several points) at the point of focus and decreaseon either side of the focal plane. Relative sharpness values can beformed from any suitable metrics, such as image contrast, resolution,entropy, variance, and/or spatial frequency content, among othermeasurements, to measure the quality of focus of images captured by amicroscope. One example equation that can be used by automatic focussystem 100 to calculate a relative sharpness value is a measure of imagevariance V, normalized by the mean μ to account for intensityfluctuations:

$V = {\frac{1}{\mu}{\sum\limits_{i = 1}^{N}\;{\sum\limits_{j = 1}^{M}\;\left\lbrack {{s\left( {i,j} \right)} - \mu} \right\rbrack^{2}}}}$where mean μ is the mean of the grayscale pixels values of all pixels,s(i,j) is the grayscale pixel value at coordinates (i,j) and N and Mrepresent the number of pixels in the i and j directions respectively.Other example methods for calculating a quality of focus value that canbe used by automatic focus system 100 are described by Sivash Yazdanfaret al., “Simple and Robust Image-Based Autofocusing for DigitalMicroscopy,” Optics Express Vol. 16, No. 12, 8670 (2008), which ishereby incorporated by reference herein in its entirety. The abovedisclosed methods are just examples and are not intended to be limiting.

FIG. 4 shows a graph comprising an X axis that represents the relativeposition of the top of stage 30 in a Z direction with respect toobjective 25 (the “relative Z position”) and a Y axis representing arelative sharpness value. The relative Z position can be changed byadjusting a stage 30 towards or away from objective 25 and/or byadjusting objective 25 towards or away from stage 30. The sharpnesscurve in FIG. 4 shows, at each relative Z position, a relative sharpnessvalue of an image captured/viewed by imaging device 5. As shown in FIG.4 , the sharpness curve can have a maximum measured sharpness (e.g., arelative sharpness value of 70 in FIG. 4 ) at a given relative Zposition (e.g., Z position 130) (that can be referred to as the in-focusposition) and may decrease symmetrically on each side of the in-focusposition (e.g., Z position 130). In some instances, the slope of thecurve in FIG. 4 at the in-focus position can be zero or close to zero.It should be understood that the term “in-focus” as used herein isintended to denote when the relative positioning of the objective andthe stage are such that a sharpness measurement is at a point at or nearthe top of a sharpness curve. The term “in-focus” is not intended to belimited to perfect or optimal focus. In-focus can be definedmathematically, by an operator and/or other suitable methods. The datashown in FIG. 4 can be compiled through a continuous measure (e.g., asharpness curve) or discrete points that can be collected by varying therelative Z position.

The range of coarse Z movement is represented by lines 137 (e.g., at 500μm) and 142 (e.g., at 2500 μm) in FIG. 4 and FIG. 5 . The range of finefocus Z movement is represented by lines 136 (e.g., at 1400 μm) and 141(e.g., at 1600 μm) in these figures. Note, that the range of Z movementrefers to a practical range of movement to achieve different Z positionsbetween objective 25 and stage 30. The range of Z movement also refersto the range of Z movement where a sharpness calculation can be used tofocus a specimen. Arrow 135 shows the sharpness score increasing to amaximum point at Z position 130 (indicating that the image is consideredto be in focus as described above) as stage 30 and objective 25 movefarther apart and arrow 140 shows the sharpness score decrease frommaximum point at Z position 130 as stage 30 and objective 25 continue tomove farther apart.

It should be apparent that the relative sharpness values and the Zpositions illustrated in FIGS. 4 and 5 are just examples and that othervalue combinations may be measured in any given application.

As mentioned above, in some instances, the slope of the curve in FIG. 4at the in-focus position can be zero or close to zero. This can makefinding a single best focus position difficult.

As described in detail below, cameras 70 and 71 can be used to help findan in-focus position of a specimen even when the slope of the curve inFIG. 4 at the in-focus position is zero or close to zero.

FIG. 5 shows sharpness curves A and B for images of a specimen taken byoffset focusing cameras 70 and 72 respectively. Similar to FIG. 4 , theX axis of the graph represents relative Z positions, the Y axisrepresents relative sharpness values and line 130 represents focusingconjugate plane 80 and indicates the Z position where the maximummeasured sharpness value for a specimen, using imaging device 5, may befound. Sharpness curve A shows, at each relative Z position, therelative sharpness value of an image captured by first offset focusingcamera 70. Sharpness curve B shows, at each relative Z position, therelative sharpness value of an image captured by second offset focusingcamera 72. FIG. 5 shows that the negative slope of curve A and thepositive slope of curve B intersect at focusing conjugate plane 80, at138. At 138, the sharpness values of images captured by each of offsetfocusing cameras 70 and 72 of a specimen at the same Z position (i.e.,1500 μm) are the same. In some embodiment, the data shown in FIG. 5 canbe compiled through a continuous measure (e.g., a sharpness curve) ordiscrete points that can be collected by varying the relative Zposition. In some embodiments, the curves in FIG. 5 are not known.

Using the properties of curves A and B for cameras 70 and 72 illustratedin FIG. 5 , system 100 can be configured to sample relative sharpnessvalues at two relative Z positions for each of cameras 70 and 72 and usethe resulting sharpness values and relative Z positions to determine thedirection of needed relative movement of the stage and the objective(i.e., whether to increase or decrease the relative Z position) in orderto bring the specimen into focus. Moreover, in some embodiments, system100 can additionally be configured to use the resulting sharpness valuesand relative Z positions to determine the amount of needed relativemovement of the stage and the objective (i.e., the amount of change inthe relative Z position) in order to bring the specimen into focus.

More particularly for example, in some embodiments, by determining theslope of a line between two relative Z positions along curve A and bydetermining the slope of a line between the same two relative Zpositions along curve B, one can determine whether the current relativeZ position is: to the left of the peak of curve A, between the peak ofcurve A and the focal point, between the focal point and the peak ofcurve B, or to the right of the peak of curve B. Upon knowing thisinformation, the direction of required movement is known. That is, ifthe current relative Z position is to the left of the peak of curve A,or between the peak of curve A and the focal point, then the relativeposition needs to increase. If the current relative Z position isbetween the focal point and the peak of curve B, or to the right of thepeak of curve B, then the relative position needs to decrease.

FIG. 6 , with further reference to FIGS. 1-5 , shows an example process600 for bringing a specimen in-focus in automatic focusing system 100,in accordance with some embodiments of the disclosed subject matter.

At 610, a specimen is placed on stage 30. Stage 30 and/or objective 25can be moved until a top surface of stage 30 is positioned at a positionZ1 within the operating range as defined by Z positions 137 and 142.

At 620, a sharpness value of the specimen at relative Z position Z1 canbe measured using each of offset focusing cameras 70 and 72. Thesharpness value SA1 for an image of specimen S, captured by focusingcamera 70, can be recorded, along with the Z1 position. The sharpnessvalue SB1 for an image of specimen S, captured by focusing camera 72 canbe recorded, along with the Z1 position.

At 630, stage 30 and/or objective 25 can be moved until specimen S is ata different relative Z position Z2 within Z positions 137 and 142. Insome embodiments, position Z2 can be selected so that the distancebetween Z1 and Z2 is twice the distance of the depth of focus (DOF) ofobjective 25 or another multiple that is less than 2 (e.g., 1.5 timesDOF, 1 times DOF or 0.5 times DOF). The disclosed method for selectingposition Z2 is just an example and not intended to be limiting.

Next, at 640, similarly to 620, a sharpness value of the specimen atrelative Z position Z2 can be measured using each of offset focusingcameras 70 and 72. The sharpness value SA2 for an image of specimen S,captured by focusing camera 70, can be recorded, along with the Z2position. The sharpness value SB2 for an image of specimen S, capturedby focusing camera 72 can be recorded, along with the Z2 position.

At 650, the slope MA1 of a line formed between (Z1,SA1) and (Z2,SA2) canbe calculated and recorded. For example, MA1 can be equal to(SA2−SA1)/(Z2−Z1). Similarly, the slope MB1 of the line formed between(Z1,SB1) and (Z2,SB2) can be calculated and recorded. For example, MB1can be equal to (SB2−SB1)/(Z2−Z1).

At 660, the relative Z position can be adjusted to bring specimen S intofocus. In some embodiments, the relative Z position can be adjusted tobring specimen S into focus in any suitable manner. For example, in someembodiments, the relative Z position can be adjusted to bring specimen Sinto focus by performing the Z adjustments listed for differentcombinations of values of MA1, MB1, SA2, and SB2 as detailed in thefollowing table.

If Value of MA1 and MB1 is: Then Make Z Adjustment MA1 is positive; andIncrease the relative Z position until the MB1 is positive or 0sharpness values for images of the specimen captured by offset focusingcameras 70 and 72 are the same (or are within any suitable tolerance ofeach other) (e.g., as indicated by point of intersection 138 in FIG. 5)and the direction of slopes MA2 and MB2 (as described below) at thepoint of intersection are opposite each other. MA1 is negative or 0;Decrease the relative Z position until the and MB1 is negative sharpnessvalues for images of the specimen captured by offset focusing cameras 70and 72 are the same (or are within any suitable tolerance of each other)(e.g., as indicated by point of intersection 138 in FIG. 5) and thedirection of slopes MA2 and MB2 (as described below) at the point ofintersection are opposite each other. MA1 is negative or 0; Increase therelative Z position until the and MB1 is positive sharpness values forimages of the specimen or 0; and The value of captured by offsetfocusing cameras 70 and SA2 is greater 72 are the same (or are withinany suitable than SB2 tolerance of each other) (e.g., as indicated bypoint of intersection 138 in FIG. 5) and the direction of slopes MA2 andMB2 (as described below) at the point of intersection are opposite eachother. MA1 is negative or 0; Decrease the relative Z position until theand MB1 is positive sharpness values for images of the specimen or 0;and The value captured by offset focusing cameras 70 and of SA2 is less72 are the same (or are within any suitable than SB2 tolerance of eachother) (e.g., as indicated by point of intersection 138 in FIG. 5) andthe direction of slopes MA2 and MB2 (as described below) at the point ofintersection are opposite each other.The point of intersection can occur when images captured by focusingcamera 70 and 72 have the same sharpness value (or are within anysuitable tolerance of each other) at the same relative Z position (asindicated by 138 in FIG. 5 ).

The disclosed table is just an example method for focusing a specimenusing offset focusing cameras 70 and 72 and is not intended to belimiting. Other methods for focusing a specimen using offset focusingcameras 70 and 72 can be used.

In performing what is described in the table above, when adjusting(i.e., increasing or decreasing) the relative Z position until thesharpness values for images of the specimen captured by offset focusingcameras 70 and 72 are the same (or are within any suitable tolerance ofeach other), system 100 can perform any suitable actions. For example,in some embodiments, system 100 can adjust the relative Z position inthe indicated direction by a given amount, repeat 620, determine if thesharpness values for images of the specimen captured by offset focusingcameras 70 and 72 are the same (or are within any suitable tolerance ofeach other), and, if the values are not the same (or within thetolerance), then repeat 630, 640, and 650 until the sharpness values forimages of the specimen captured by offset focusing cameras 70 and 72 arethe same (or are within any suitable tolerance of each other) (e.g., asindicated by point of intersection 138 in FIG. 5 ) and the direction ofslopes MA2 and MB2 (as described below) at the point of intersection areopposite each other.

Any suitable given amount(s) of movement can be used at 650, and thegiven amount(s) can be determined in any suitable manner. For example,in some embodiments, the given amount can always be a fixed amount. Thisfixed amount can be determined based on the configurations of system 100and can be user specified. As another example, in some embodiments, thegiven amount can vary based on the slopes determined at 640. Moreparticularly, the given amount can be larger when the slopes indicatethat the relative Z position is to the left of the peak of curve A inFIG. 5 or to the right of the peak of curve B in FIG. 5 , and the givenamount can be smaller when the slopes indicate that the relative Zposition is between the peak of curve A in FIG. 5 and the peak of curveB in FIG. 5 . As still another example, the given amount can be based onthe slopes determined at 640 and be a function of the values of SA1 andSB1 determined at 620. More particularly, when the slopes indicate thatthe relative Z position is to the left of the peak of curve A in FIG. 5or to the right of the peak of curve B in FIG. 5 , the given amount canbe inversely proportional to the difference between SA1 and SB1 (i.e.,when the difference between SA1 and SB1 is small, the given amount willbe large, and when the difference between SA1 and SB1 is large, thegiven amount will be small), and when the slopes indicate that therelative Z position is between the peak of curve A in FIG. 5 and thepeak of curve B in FIG. 5 , the given amount can be proportional to thedifference between SA1 and SB1 (i.e., when the difference between SA1and SB1 is small, the given amount will be small, and when thedifference between SA1 and SB1 is large, the given amount will be large)(in this example, the given amount can be prevented from dropping belowa minimum value so that intersection 138 can be found quickly).

As indicated in the table above, when adjusting the relative Z positionto bring specimen S into focus, process 600 can determine whether thedirection of slopes MA2 and MB2 are opposite each other. To determinethe direction of the slopes MA2 and MB2 at the point of intersection,stage 30 and/or objective 25 can first be moved from the point ofintersection until specimen S is at a different position Z3 within Zpositions 137 and 142. In some embodiments, position Z3 can be selectedso that its distance from the point of intersection is twice the depthof focus (DOF) of objective 25 or another multiple that is less than 2(e.g., 1.5 times DOF, 1 times DOF or 0.5 times DOF). The disclosedmethod for selecting position Z3 is just an example and not intended tobe limiting.

Next, sharpness values SA3 and SB3 and Z3 position can be recorded forimages of the specimen captured by offset focusing cameras 70 and 72.

Then, the slope MA2 of a line formed between the point of intersectionand (Z3,SA3), and the slope MB2 of a line formed between the point ofintersection and (Z3,SB3), can be calculated.

If the direction of the slopes MA2 and MB2 are opposite to each other,then the point of intersection is at the image-forming conjugate plane.Otherwise, the stage and/or objective can be continued to be adjustedaccording to the instructions in the table above.

In some embodiments, the sharpness value at the point of intersectioncan be compared to a recorded sharpness setpoint for automatic focussystem 100, a particular specimen, specimen class and/or any othersuitable classification group. If the sharpness value at the point ofintersection is the same or within an acceptable variance of therecorded sharpness value, then the point of intersection occurs when thepoint of intersection is at the image-forming conjugate plane.Otherwise, the stage and/or objective can be continued to be adjustedaccording to the instructions in the table above.

In some embodiments, once the relative Z position where the specimen isin-focus is determined, the absolute position of: stage 30; objective25; the top of the specimen on stage 30; and/or the distance between thetop of stage 30 and objective 25, can be stored by control system 108 asa position setpoint. The position setpoint can be associated with aparticular specimen, a particular specimen class and/or any othersuitable classification group for the specimen.

Because the slopes, along with certain points, on sharpness curves A andB indicate whether the distance between the stage and objective must bedecreased or increased to bring the specimen in focus, fewer images ofthe specimen can be taken to bring a specimen into focus.

At 670, in some embodiments, once the specimen is determined to bein-focus, an in-focus image can be captured by imaging device 5.

In some embodiments, once the specimen is determined to be in-focususing the method described in 610-660, imaging device 5 can be used tofine tune the focus of the specimen. For example, using imaging device5, sharpness values of the specimen can be calculated for at least tworelative Z positions of the stage and objective to determine whether anestimated maximum sharpness has been achieved or the relative Z-positionneeds to be adjusted to achieve an estimated maximum sharpness (i.e.,the point on the sharpness curve where the slope is 0 or close to 0).

In some embodiments, control system 108 can also determine whether thereis a position setpoint associated with the specimen, specimen classand/or any other suitable classification group for the specimen placedon stage 30, and can position autofocus system 100 at that positionsetpoint at 610. Knowing an approximate target relative Z position,reduces the relative Z distance that is needed to focus the specimen andallows the offset focusing cameras to be positioned closer to focusingconjugate plane 80. As discussed above, the slope of the sharpnesscurves become steeper as offset focusing cameras 70 and 72 move closerto focusing conjugate plane 80. A steeper slope, represents greaterresolution or a larger change in sharpness versus a smaller change in Zheight. A steeper slope can allow for finer focal adjustment andcontrol.

The division of when the particular portions of process 600 areperformed can vary, and no division or a different division is withinthe scope of the subject matter disclosed herein. Note that, in someembodiments, blocks of process 600 can be performed at any suitabletimes. It should be understood that at least some of the portions ofprocess 600 described herein can be performed in any order or sequencenot limited to the order and sequence shown in and described in FIG. 6in some embodiments. Also, some of the portions of process 600 describedherein can be or performed substantially simultaneously whereappropriate or in parallel in some embodiments. Additionally oralternatively, some portions of process 600 can be omitted in someembodiments.

Process 600 can be implemented in any suitable hardware and/or software.For example, in some embodiments, process 600 can be implemented incontrol system 108 shown in FIG. 1 .

The location of focusing conjugate plane 80 can be determined in anysuitable manner in some embodiments. For example, in some embodiments,focusing conjugate plane 80 can be determined mathematically based oncharacteristics of automatic focus system 100.

As another example, in some embodiments, focusing conjugate plane 80 canbe determined using a calibration process. FIG. 7 , with furtherreference to FIGS. 1-6 , shows an example of a calibration process 700for finding focusing conjugate plane 80 and calibrating offset distancesf1 and f2 for offset focusing cameras 70 and 72 respectively, inaccordance with some embodiments of the disclosed subject matter. Insome embodiments, focusing conjugate plane 80 can be determinedmathematically based on characteristics of automatic focus system 100.In other embodiments, focusing conjugate plane 80 can be determinedexperimentally, for example, by generating a set of sharpness curves forone of offset focusing cameras 70 and 72 as described, for example, in710-740.

After process 700 begins, at 710, specimen S can be placed on stage 30and imaging device 5 can be used to determine and record when therelative Z position for when the specimen is in-focus, as discussedabove in connection with FIG. 4 . For example, FIG. 4 shows a specimento be in focus at a relative Z position of 1500 μm (represented by line130), which is at the peak of the curve shown in FIG. 4 . Process 700can find this relative Z position by stepping through any suitablenumber of Z positions in the range of Z positions shown by lines 137 and142, capturing an image with imaging device 5 at each position, anddetermining a relative sharpness value (as described above). Thishighest of these sharpness values can be determined to correspond to thein-focus point.

At 720, offset focusing camera 70 can be moved incrementally in ahorizontal direction and a sharpness curve, or a set of sharpness valuesrelative to different relative Z positions, can be calculated at eachhorizontal position of the offset focusing camera. The position of theoffset focusing camera where the peak of its sharpness curve, or themaximum sharpness value, occurs can be defined as being at the positionat which focusing conjugate plan 80 lies as shown in FIG. 1 . Thisposition can be defined as being the in-focus relative Z position forcamera 70 and can be recorded. In some embodiments, the maximumsharpness value at the in-focus relative Z position for camera 70 can bestored by control system 108 as the sharpness setpoint for automaticfocus system 100, a particular specimen, specimen class and/or any othersuitable classification group.

At 725, the in-focus relative Z position for camera 72 can be the sameas the in-focus relative Z position for camera 70 (e.g., when focusinglenses 22 and 24 are the same). In such cases, camera 72 can be simplymoved to the same in-focus relative Z position (which is at focusingconjugate plane 80) as camera 70.

In some embodiments, however, the in-focus relative Z position forcamera 72 can be different from the in-focus relative Z position forcamera 70. In such cases, offset focusing camera 72 can be movedincrementally in a horizontal direction and a sharpness curve, or a setof sharpness values relative to different relative Z positions, can becalculated at each horizontal position of the offset focusing camera.The position of the offset focusing camera where the peak of itssharpness curve, or the maximum sharpness value, occurs can be definedas being at the position at which a focusing conjugate plane lies (notshown in the figures). This position can be defined as being thein-focus relative Z position for camera 72 and can be recorded. In someembodiments, the maximum sharpness value at the in-focus relative Zposition for camera 72 can be stored by control system 108 as thesharpness setpoint for camera 72.

While 720 discusses performing actions for camera 70 and 725 discussesperforming actions for camera 72, the cameras for which action are takenin 70 and 72 can be swapped in some embodiments.

At 730, offset focusing camera 70 can be positioned at an offsetdistance f1 on a first side (e.g., positive, negative, left, right, top,bottom) of its in-focus relative Z position. Any suitable offsetdistance value can be used in some embodiments. For example, in someembodiments, offset focusing camera 70 can initially be positioned at 30cm, 15 cm, 10 cm, and/or any other suitable offset distance from itsin-focus relative Z position.

At 735, offset focusing camera 72 can be positioned at an offsetdistance f2 on a second, opposite side (e.g., negative, positive, right,left, bottom, top, respectively) of its in-focus relative Z position.That is, for example, if camera 70 is positioned on the left of itsin-focus relative Z position, then camera 72 is positioned on the rightside of its in-focus relative Z position, and vice versa. Any suitableoffset distance value can be used in some embodiments. For example, insome embodiments, offset focusing camera 72 can initially be positionedat the same offset distance from its in-focus relative Z position asoffset focusing camera 70 is from its in-focus relative Z position(e.g., 30 cm, 15 cm, 10 cm, and/or any other suitable offset distancefrom its in-focus relative Z position).

In some embodiments, the initial offset distances for f1 and f2 do nothave to be equal to each other. The initial offset distances can bebased on optical characteristics of automatic focus system 100 and/orprecision requirements for focusing. The closer offset focusing camera70 or 72 is placed to focusing conjugate plane 80, the steeper the slopeof its sharpness curve at focusing conjugate plane 80. A steeper sloperepresents a larger change in sharpness versus a smaller change in Zheight (also referred to as greater resolution). A steeper slope can bedesirable because it allows for finer focal adjustment and control.

More specifically, if the range of Z movement (e.g., the distancebetween lines 137 and 142 shown in FIG. 6 ) is larger, then offsetfocusing cameras 70 and 72 can be positioned farther away from focusingconjugate plane 80 than if the range of Z movement is smaller.

In some embodiments, offset distances f1 and f2 can also be based on thesteepness of the sharpness curve at the Z position where the specimen isin optimum focus for imaging device 5. For example, if the range of Zmovement necessary to bring a specimen in focus is small (e.g., between1300 μm-1700 μm), then offset focusing cameras 70 and 72 can be placedcloser to focusing conjugate plane 80, because a closer position has asteeper slope and greater resolution compared to an offset distancefarther away from focusing conjugate plane 80. In some embodiments,offset distances f1 and f2 farther away from focusing conjugate plane 80can be selected to accommodate a maximum range of Z movement forautomatic focus system 100, so that the offset focusing cameras 70 and72 do not have to constantly be repositioned for specimens of varyingthicknesses.

At 740, one of offset focusing cameras 70 and 72 can remain in a fixedposition and the other offset focusing camera can be repositioned untilthe sharpness curves (e.g., sharpness curves A and B shown in FIG. 5 )calculated for offset focusing cameras 70 and 72, respectively,intersect at focusing conjugate plane 80. For example, second offsetfocusing camera 72 can remain in a fixed position, offset from focusingconjugate plane 80, and a sharpness curve for specimen S, using secondoffset focusing camera 72, can be calculated and recorded. Inparticular, the sharpness value for specimen S (e.g., 28) at in-focus Zposition 130 (e.g., 1500 μm), using offset focusing camera 72, can berecorded. Then first offset focusing camera 70 can be movedincrementally in a horizontal direction, towards or away from focusingconjugate plane 80 until it is determined that sharpness curve Aintersects with sharpness curve B at in-focus Z position 130 (e.g., asshown in FIG. 5 ), which lies on focusing conjugate plane 80. The slopeswill intersect when the sharpness value for a specimen at in-focus Zposition 130, using first focusing camera 70, is the same value as usingsecond offset focusing camera 72 (e.g., 28) at in-focus Z position 130.At the point of intersection, the slopes of sharpness curves A and B(i.e., any slope encompassing the point of intersection) are opposite toeach other. Sharpness curve A can be calculated and recorded at theposition of intersection. In some embodiments, the recorded sharpnesscurves A and B can be used to evaluate the relationship betweendifferent points on curves A and B (i.e., points other than the point ofintersection) relative to focusing conjugate plane 80 and used toanalyze a specimen or to set the focus at a plane other than theconjugate plane.

In some embodiment, the relative sharpness value at the intersection ofcurves A and B preferably do not occur at the very top or the verybottom of the sharpness curve. Rather, in some embodiments, theintersection can occur in any suitable range within the minimum andmaximum values of the sharpness curves, such as between 10-90% ofminimum and maximum values of the sharpness curves, between 5-95% ofminimum and maximum values of the sharpness curves, between 20-80% ofminimum and maximum values of the sharpness curves, etc.

In other embodiments, both offset focusing cameras 70 and 72 can bemoved towards and/or away from focusing conjugate plane 80, untilsharpness curve A intersects with curve B at in-focus Z position 130.Sharpness curves A and B can be calculated, for example, as described inconnection with FIG. 4 . The disclosed methods for calibrating offsetdistances f1 and f2 are just examples and are not intended to belimiting.

In some embodiments, sharpness curves do not need to be calculated forfocusing cameras A and B to find offset distances f1 and f2. Forexample, one of offset focusing cameras 70 or 72 can be moved until thesharpness values calculated for images of a specimen captured byfocusing cameras A and B at in-focus Z position 130 are equal.

In some embodiments, offset distances f1 and f2 can be set once forautomatic focus system 100. In other embodiments, offset distances f1and f2 can be recalibrated to accommodate different objectives,different specimen thicknesses, different specimen classes or anysuitable criteria. For example, offset distances f1 and f2 can bedecreased for higher magnification objectives to accommodate a smallerdepth of field (focus) and smaller range of Z movement. In someembodiments, offset distances f1 and f2 can be saved by control system108 as offset distance setpoints. The offset distance setpoints can beassociated, for example, with the thickness and/or any other suitablecharacteristic of a specimen, the specimen class and/or any othersuitable grouping of the specimen, and/or the optical characteristics ofthe microscope (e.g., the magnification of an objective). The offsetdistance setpoints can be used to automatically position offset focusingcameras 70 and 72.

In some embodiments, a specimen class can be defined based on specimensmade from materials of similar reflective qualities.

The division of when the particular portions of process 700 areperformed can vary, and no division or a different division is withinthe scope of the subject matter disclosed herein. Note that, in someembodiments, blocks of process 700 can be performed at any suitabletimes. It should be understood that at least some of the portions ofprocess 700 described herein can be performed in any order or sequencenot limited to the order and sequence shown in and described in the FIG.7 in some embodiments. Also, some of the portions of process 700described herein can be or performed substantially simultaneously whereappropriate or in parallel in some embodiments. Additionally oralternatively, some portions of process 500 can be omitted in someembodiments.

Process 700 can be implemented in any suitable hardware and/or software.For example, in some embodiments, process 700 can be implemented incontrol system 108.

In some embodiments, control system 108 can collect data relating to theoperation and/or configuration of auto focus system 100. For example,the data can include details regarding the configuration of auto focussystem 100 when a specimen (or a portion of the specimen) is in-focussuch as: the position of offset focusing cameras 70 and 72; the absoluteposition of: stage 30, objective 25, the top of the specimen on stage30, and/or the distance between the top of stage 30 and objective 25;the sharpness setpoint, the position setpoint; the sharpness measurementused to determine that a specimen (or an area of a specimen) isin-focus. Further, data relating to the operation of automatic focus,such as the time it took to achieve focus for each area of a specimenthat is scanned, the number of images captured before a specimen (or anarea of a specimen) was determined to be in focus, the total distancestage 30 and/or objective 25 had to travel to achieve focus, therecorded sharpness value. Data regarding the specimen, specimen classand/or any other suitable classification group can also be collected.The foregoing are just examples and are not intended to be limiting ofthe type of data that can be collected.

The data collected can further be analyzed by control system 108 and/ora remote computing device coupled to control system 108 to identifyinefficiencies in the auto focus configuration and/or operation. Controlsystem 108 and/or a remote computing device can further determinewhether the inefficiencies detected correlate with an aspect of theconfiguration and/or operation of autofocus system 100 and makeappropriate adjustments to the configuration and/or operation ofautofocus system 100.

In some embodiments, any suitable artificial intelligence algorithm(s)can be used to identify inefficiencies and/or make suitable adjustments.For example, in some embodiments, the artificial intelligence algorithmscan include one or more of the following, alone or in combination:machine learning; hidden Markov models; recurrent neural networks;convolutional neural networks; Bayesian symbolic methods; generaladversarial networks; support vector machines; and/or any other suitableartificial intelligence.

Note that, in some embodiments, the techniques described herein forfocusing a specimen can apply to focusing an entire specimen or tofocusing just a portion of a specimen.

The provision of the examples described herein (as well as clausesphrased as “such as,” “e.g.,” “including,” and the like) should not beinterpreted as limiting the claimed subject matter to the specificexamples; rather, the examples are intended to illustrate only some ofmany possible aspects. It should also be noted that, as used herein, theterm mechanism can encompass hardware, software, firmware, or anysuitable combination thereof.

Some portions of above description present the features of the presentdisclosure in terms of algorithms and symbolic representations ofoperations on information. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. These operations, while describedfunctionally or logically, are understood to be implemented by computerprograms. Furthermore, it has also proven convenient at times, to referto these arrangements of operations as modules or by functional names,without loss of generality.

Unless specifically stated otherwise as apparent from the abovediscussion, it is appreciated that throughout the description,discussions utilizing terms such as “determining,” “displaying,” or thelike, refer to the action and processes of a computer system, or similarelectronic computing device, that manipulates and transforms datarepresented as physical (electronic) quantities within the computersystem memories or registers or other such information storage,transmission or display devices.

Certain aspects of the present disclosure include process steps andinstructions described herein in the form of an algorithm. It should benoted that the process steps and instructions of the present disclosurecould be embodied in software, firmware or hardware, and when embodiedin software, could be downloaded to reside on and be operated fromdifferent platforms used by real time network operating systems.

The present disclosure also relates to an apparatus for performing theoperations herein. This apparatus may be specially constructed for therequired purposes, or it may comprise a general-purpose computerselectively activated or reconfigured by a computer program stored on acomputer readable medium that can be accessed by the computer. Such acomputer program may be stored in a computer readable storage medium,such as, but is not limited to, any type of disk including floppy disks,optical disks, CD-ROMs, magnetic-optical disks, read-only memories(ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic oroptical cards, application specific integrated circuits (ASICs), or anytype of non-transient computer-readable storage medium suitable forstoring electronic instructions. Furthermore, the computers referred toin the specification may include a single processor or may bearchitectures employing multiple processor designs for increasedcomputing capability.

The algorithms and operations presented herein are not inherentlyrelated to any particular computer or other apparatus. Variousgeneral-purpose systems may also be used with programs in accordancewith the teachings herein, or it may prove convenient to construct morespecialized apparatus to perform the required method steps. The requiredstructure for a variety of these systems will be apparent to those ofskill in the art, along with equivalent variations. In addition, thepresent disclosure is not described with reference to any particularprogramming language. It is appreciated that a variety of programminglanguages may be used to implement the teachings of the presentdisclosure as described herein, and any references to specific languagesare provided for disclosure of enablement and best mode of the presentdisclosure.

The automatic microscopic focus system and method have been described indetail with specific reference to these illustrated embodiments. It willbe apparent, however, that various modifications and changes can be madewithin the spirit and scope of the disclosure as described in theforegoing specification, and such modifications and changes are to beconsidered equivalents and part of this disclosure. The scope of theinvention is limited only by the claims that follow.

What is claimed is:
 1. A method for automatically focusing a microscope,comprising: receiving, by a computing system from a first camera of themicroscope, a first image of a specimen positioned on a stage of themicroscope at a first position, wherein the first camera comprises afirst image sensor, the first camera positioned at a first offsetdistance to an image forming conjugate plane; receiving, by thecomputing system from a second camera of the microscope, a second imageof the specimen positioned on the stage of the microscope at the firstposition, wherein the second camera comprises a second image sensor, thesecond camera positioned at a second offset distance to the imageforming conjugate plane; measuring, by the computing system, a firstsharpness value of the specimen based on the first image and a secondsharpness value of the specimen based on the second image; causing, bythe computing system, adjustment of the stage of the microscope to asecond position distinct from the first position; receiving, by thecomputing system from the first camera, a third image of the specimenpositioned on the stage of the microscope at the second position;receiving, by the computing system from the second camera of themicroscope, a fourth image of the specimen positioned on the stage ofthe microscope at the second position; measuring, by the computingsystem, a third sharpness value of the specimen based on the third imageand a fourth sharpness value of the specimen based on the fourth image;and focusing, by the computing system, the first camera and the secondcamera by adjusting a relative distance between the first position andthe second position, until images generated by the first camera and thesecond camera are within a threshold sharpness level.
 2. The method ofclaim 1, further comprising: generating, by the computing system, afirst line between the first image and the third image, wherein thefirst position and the first sharpness value are a first set ofcoordinates and the second position and the third sharpness value are asecond set of coordinates; and calculating, by the computing system, afirst slope of the first line.
 3. The method of claim 2, furthercomprising: generating, by the computing system, a second line betweenthe second image and the fourth image, wherein the first position andthe second sharpness value are a third set of coordinates and the secondposition and the fourth sharpness value are a fourth set of coordinates;and calculating, by the computing system, a second slope of the secondline.
 4. The method of claim 3, wherein focusing, by the computingsystem, the first camera and the second camera by adjusting the relativedistance between the first position and the second position, until theimages generated by the first camera and the second camera are withinthe threshold sharpness level comprises: increasing the relativedistance between the first position and the second position based on thefirst slope and the second slope being non-negative.
 5. The method ofclaim 3, wherein focusing, by the computing system, the first camera andthe second camera by adjusting the relative distance between the firstposition and the second position, until the images generated by thefirst camera and the second camera are within the threshold sharpnesslevel comprises: decreasing the relative distance between the firstposition and the second position based on the second slope beingnegative.
 6. The method of claim 3, wherein focusing, by the computingsystem, the first camera and the second camera by adjusting the relativedistance between the first position and the second position, until theimages generated by the first camera and the second camera are withinthe threshold sharpness level comprises: increasing the relativedistance between the first position and the second position based on thefirst slope being non-positive, the second slope being non-negative, andthe third sharpness value is greater than the fourth sharpness value. 7.The method of claim 3, wherein focusing, by the computing system, thefirst camera and the second camera by adjusting the relative distancebetween the first position and the second position, until imagesgenerated by the first camera and the second camera are within thethreshold sharpness level comprises: decreasing the relative distancebetween the first position and the second position based on the firstslope being non-positive, the second slope being non-negative, and thethird sharpness level is less than the fourth sharpness level.
 8. Asystem, comprising: a processor in communication with a microscope; anda memory having programming instructions stored thereon, which, whenexecuted by the processor, performs one or more operations, comprising:receiving, from a first camera of the microscope, a first image of aspecimen positioned on a stage of the microscope at a first position,wherein the first camera comprises a first image sensor, the firstcamera positioned at a first offset distance to an image formingconjugate plane; receiving, from a second camera of the microscope, asecond image of the specimen positioned on the stage of the microscopeat the first position, wherein the second camera comprises a secondimage sensor, the second camera positioned at a second offset distanceto the image forming conjugate plane; measuring a first sharpness valueof the specimen based on the first image and a second sharpness value ofthe specimen based on the second image; causing adjustment of the stageof the microscope to a second position distinct from the first position;receiving, from the first camera, a third image of the specimenpositioned on the stage of the microscope at the second position;receiving, from the second camera of the microscope, a fourth image ofthe specimen positioned on the stage of the microscope at the secondposition; measuring, by a computing system, a third sharpness value ofthe specimen based on the third image and a fourth sharpness value ofthe specimen based on the fourth image; and focusing the first cameraand the second camera by adjusting a relative distance between the firstposition and the second position, until images generated by the firstcamera and the second camera are within a threshold sharpness level. 9.The system of claim 8, wherein the one or more operations furthercomprise: generating a first line between the first image and the thirdimage, wherein the first position and the first sharpness value are afirst set of coordinates and the second position and the third sharpnessvalue are a second set of coordinates; and calculating a first slope ofthe first line.
 10. The system of claim 9, wherein the one or moreoperations further comprise: generating a second line between the secondimage and the fourth image, wherein the first position and the secondsharpness value are a third set of coordinates and the second positionand the fourth sharpness value are a fourth set of coordinates; andcalculating a second slope of the second line.
 11. The system of claim10, wherein focusing the first camera and the second camera by adjustingthe relative distance between the first position and the secondposition, until the images generated by the first camera and the secondcamera are within the threshold sharpness level comprises: increasingthe relative distance between the first position and the second positionbased on the first slope and the second slope being non-negative. 12.The system of claim 10, wherein focusing the first camera and the secondcamera by adjusting the relative distance between the first position andthe second position, until the images generated by the first camera andthe second camera are within the threshold sharpness level comprises:decreasing the relative distance between the first position and thesecond position based on the second slope being negative.
 13. The systemof claim 10, wherein focusing the first camera and the second camera byadjusting the relative distance between the first position and thesecond position, until the images generated by the first camera and thesecond camera are within the threshold sharpness level comprises:increasing the relative distance between the first position and thesecond position based on the first slope being non-positive, the secondslope being non-negative, and the third sharpness value is greater thanthe fourth sharpness value.
 14. The system of claim 10, wherein focusingthe first camera and the second camera by adjusting the relativedistance between the first position and the second position, until theimages generated by the first camera and the second camera are withinthe threshold sharpness level comprises: decreasing the relativedistance between the first position and the second position based on thefirst slope being non-positive, the second slope being non-negative, andthe third sharpness value is less than the fourth sharpness value.
 15. Anon-transitory computer readable medium including one or more sequencesof instructions, which, when executed by one or more processors, causesthe one or more processors to perform operations, comprising: receiving,by a computing system from a first camera of a microscope, a first imageof a specimen positioned on a stage of the microscope at a firstposition, wherein the first camera comprises a first image sensor, thefirst camera positioned at a first offset distance to an image formingconjugate plane; receiving, by the computing system from a second cameraof the microscope, a second image of the specimen positioned on thestage of the microscope at the first position, wherein the second cameracomprises a second image sensor, the second camera positioned at asecond offset distance to the image forming conjugate plane; measuring,by the computing system, a first sharpness value of the specimen basedon the first image and a second sharpness value of the specimen based onthe second image; causing, by the computing system, adjustment of thestage of the microscope to a second position distinct from the firstposition; receiving, by the computing system from the first camera, athird image of the specimen positioned on the stage of the microscope atthe second position; receiving, by the computing system from the secondcamera of the microscope, a fourth image of the specimen positioned onthe stage of the microscope at the second position; measuring, by thecomputing system, a third sharpness value of the specimen based on thethird image and a fourth sharpness value of the specimen based on thefourth image; and focusing, by the computing system, the first cameraand the second camera by adjusting a relative distance between the firstposition and the second position, until images generated by the firstcamera and the second camera are within a threshold sharpness level. 16.The non-transitory computer readable medium of claim 15, furthercomprising: generating, by the computing system, a first line betweenthe first image and the third image, wherein the first position and thefirst sharpness value are a first set of coordinates and the secondposition and the third sharpness value are a second set of coordinates;and calculating, by the computing system, a first slope of the firstline.
 17. The non-transitory computer readable medium of claim 16,further comprising: generating, by the computing system, a second linebetween the second image and the fourth image, wherein the firstposition and the second sharpness value are a third set of coordinatesand the second position and the fourth sharpness value are a fourth setof coordinates; and calculating, by the computing system, a second slopeof the second line.
 18. The non-transitory computer readable medium ofclaim 17, wherein focusing, by the computing system, the first cameraand the second camera by adjusting the relative distance between thefirst position and the second position, until the images generated bythe first camera and the second camera are within the thresholdsharpness level comprises: increasing the relative distance between thefirst position and the second position based on the first slope and thesecond slope being non-negative.
 19. The non-transitory computerreadable medium of claim 17, wherein focusing, by the computing system,the first camera and the second camera by adjusting the relativedistance between the first position and the second position, until theimages generated by the first camera and the second camera are withinthe threshold sharpness level comprises: decreasing the relativedistance between the first position and the second position based on thesecond slope being negative.
 20. The non-transitory computer readablemedium of claim 17, wherein focusing, by the computing system, the firstcamera and the second camera by adjusting the relative distance betweenthe first position and the second position, until the images generatedby the first camera and the second camera are within the thresholdsharpness level comprises: increasing the relative distance between thefirst position and the second position based on the first slope beingnon-positive, the second slope being non-negative, and the thirdsharpness value is greater than the fourth sharpness value.